|
|
|
| e-Pub |
Previous | 
Home
Section: New Results
Novel interactive technologies
Participants : Géry Casiez, Fanny Chevalier, Stéphane Huot [correspondent] , Sylvain Malacria, Thomas Pietrzak, Sébastien Poulmane, Thibault Raffaillac, Nicolas Roussel, Marcelo Wanderley.
Technologies for prototyping, tuning and studying interactive systems
Measure, characterization and compensation of end-to-end latency
We developed a low-cost method to measure and characterize the end-to-end latency when using a touch system (tap latency) or an input device equipped with a physical button [22]. Our method relies on a vibration sensor attached to a finger and a photo-diode to detect the screen response (see Fig. 5). Both are connected to a micro-controller connected to a host computer using a low-latency USB communication protocol in order to combine software and hardware probes to help determine where the latency comes from. We presented the operating principle of our method and investigated the main sources of latency in several systems. We showed that most of the latency originates from the display side. Our method can help application designers characterize and troubleshoot latency on a wide range of interactive systems.
|
When there is no other way to reduce latency (e.g. hardware improvements or software optimization), we have to rely on latency compensation methods. In direct interaction, this problem is considered and formulated as a trajectory prediction problem, i.e. where should the active position be without latency? We addressed this problem by constructing a frequency-domain approximation of the non-casual ideal predictor [19]. This approximation can be computed analytically, or obtained as an optimization task. We also proposed an adaptive modification of the algorithm taking into account possible variations in user behavior, and we empirically illustrated its applicability.
Turning function calls into animations
Animated transitions are an integral part of modern interaction frameworks. While early systems would animate a few properties, such as position or color, with different functions for each, modern systems contain too many properties to scale this way. To cope with this increasing numbers of properties to animate, most frameworks rely on naming strategies to refer to properties – like “position” or “color” – instead of having one specific function for each. This improves flexibility for choosing animated properties at runtime and reduces API size. It also gives an implicit contract that any property can animate. This flexibility has a price though. The animation of custom properties and types requires frameworks to provide an advanced API that exposes low-level details of their animation systems (e.g., timers and threads). This results in larger APIs and cumbersome syntaxes owing to the complex underlying mechanisms. It also creates a steep learning curve from basic to advanced APIs, which is likely to force programmers to stick to existing animatable properties whenever possible.
We have studied better ways for animating all objects in the system, independently of any frameworks, and aimed for a syntax reusing as much as possible the existing lexical elements [33]. This resulted in a delay operator appended to function calls – object.setProperty(target) during 2s – that turns setters into animations. It offers a coherent way to express animations across frameworks, and facilitates the animation of new properties. Our proof-of-concept implementation has been done in the Smalltalk language and tested with three popular interaction frameworks for the Pharo platform. In the future, this prototype should contribute to measuring the effect of animation syntax on programmers’ ease in prototyping.
Interactive tuning of parameters without code recompilation
We developed libParamTuner, a cross-platform library that allows interactive tuning of parameters in applications written in C++ or Java, without the need to re-compile [20]. libParamTuner provides a lightweight syntax to bind some variables of an application to parameters defined in an XML file. Each modification of the XML file updates in real time the associated parameters in the application. A graphical interface allows editing the XML file, using interactive controls dynamically created for each parameter.
Technologies for creative applications
Leveraging flexion input for pen interaction
For artists, digital media offers some advantages over physical media, such as non-destructive editing, automation, and potential novel effects. However, conveying artistic intent to a computer system can be problematic. An artist working in traditional media, such as a painter or pastellist, can subtly manipulate their artistic tool to create different effects, while a digital artist working with a non-augmented stylus is only able to indicate a path, shape, or point on the surface of the tablet. Augmented styluses help solve this problem by using physical inputs such as pressure or tilt to function as parametric controls. We designed and developed FlexStylus, a flexible stylus that detects deformation of the barrel as a vector with both a rotational and an absolute value [24]. It provides two additional degrees of freedom with the goal of improving the expressiveness of digital art using a stylus device. The prototype uses a cluster of four fiber-optic-based deformation sensors. We also proposed interaction techniques using the FlexStylus to improve menu navigation and tool selection (see Fig. 6). Finally, we conducted a study comparing users' ability to match a changing target value using a commercial pressure stylus and the FlexStylus' absolute deformation. When using the FlexStylus, users had higher accuracy overall, suggesting that deformation may be a useful input method for future work considering stylus augmentation.
|
Versioning and annotation support for collaborative mapping design
A crucial component of the digital musical instrument (DMI) building process involves the mapping of sensor or gesture input signals from the musician to relevant synthesis parameters. While there are many different approaches to the design and implementation of DMI mapping, for many designers, a common part of the process involves the use of graphical user interfaces where the connection of signals can be observed and manipulated visually. In that context, and especially during collaborative design, being able to effectively store, preview, and reload previous configurations can reduce risks, encourage exploration, and improve the management of configuration files. We developed a new mapping tool for the libmapper library, a well-used system for connecting signals employed for the design and implementation of DMIs, that deeply integrate graphical versioning and comparison tool, and rich annotation features that go beyond the text-based tagging commonly employed by existing versioning systems [36]. We plan to further extend this approach with richer annotation capabilities (e.g. media files or examples of use) and query features, and with additional visualizations of the mappings and their changes over time.
Probatio: a method and toolkit for generating ideas and prototypes
We have investigated aspects of DMI design by focusing on the complexity of the design space and the importance of prototyping cycles. In particular, we focused on how to provide an initial path for generating DMI ideas and how to reduce the time and effort required to build functional DMI prototypes. We proposed a new methodology and an associated physical prototyping toolkit, Probatio (see Fig. 7), which has building blocks inspired by existing instruments [13]. A preliminary user study with musicians and DMI designers revealed a strong potential for its use in the development of DMIs. It also highlighted possible improvements such as more ergonomic shapes for the building blocks and better sound mapping possibilities.
|
